Skip to content

Implement KServe deployment for NIMService#547

Merged
shivamerla merged 2 commits intoNVIDIA:mainfrom
xieshenzh:nim-kserve-xz
Jul 1, 2025
Merged

Implement KServe deployment for NIMService#547
shivamerla merged 2 commits intoNVIDIA:mainfrom
xieshenzh:nim-kserve-xz

Conversation

@xieshenzh
Copy link
Copy Markdown
Contributor

@xieshenzh xieshenzh commented Jun 23, 2025

This PR implements KServe deployment for NIMService.

Changes:

  • Add implementation to kserve/nimservice.go

Notes:

  • HPA, ServiceMonitor, RollingUpdate, SecurityContext are not supported for Serverless mode
  • KServe will discontinue Serverless mode in near future

@copy-pr-bot
Copy link
Copy Markdown

copy-pr-bot Bot commented Jun 23, 2025

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

@shivamerla
Copy link
Copy Markdown
Collaborator

thanks @xieshenzh for the PR!. Can you add some samples under config/samples/nim/kserve for both RawDeployment mode and using Knative mode?

@xieshenzh xieshenzh force-pushed the nim-kserve-xz branch 2 times, most recently from 46ac5f5 to 380a408 Compare June 28, 2025 03:58
@xieshenzh
Copy link
Copy Markdown
Contributor Author

thanks @xieshenzh for the PR!. Can you add some samples under config/samples/nim/kserve for both RawDeployment mode and using Knative mode?

I have added samples for rawdeployment and serverless. Thanks.

@shivamerla
Copy link
Copy Markdown
Collaborator

@xieshenzh Can you mark this PR accordingly when this is ready to merge? There are lot of code duplication here, but we will take care of refactoring after the merge.

Comment thread internal/controller/nimservice_controller.go Outdated
Comment thread internal/controller/platform/kserve/nimservice.go
Signed-off-by: Xieshen Zhang <xiezhang@redhat.com>
@xieshenzh xieshenzh marked this pull request as ready for review July 1, 2025 03:38
@shivamerla shivamerla merged commit 4dcedf0 into NVIDIA:main Jul 1, 2025
9 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants